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Abstract— THIS PAPER IS ELIGIBLE FOR THE STUDENT 
PAPER AWARD. In this paper, we study relay networks with 
orthogonal components in presence of noncausal channel state 
information (CSI) available at the source. We propose an upper 
bound on the capacity of the discrete memoryless model (DM) 
for the case in which just the source component intended for 
the destination is encoded against the CSI known non-causally 
at the source. Also, we derive capacity for two special classes of 
the Gaussian structure of the model. The first class is the one for 
which we have obtained the upper bound and the second class is 
the one in which all of the source components intended for the 
relays and destination are encoded against the noncausal CSI, 
however, no interference at the relays and destination exists in 
this case. 

I. Introduction 

Relay networks are one of the basic structures in wireless 
communication networks (such as ad-hoc networks) that im- 
prove the total transmission rate and coverage region. Relay 
networks have been extensively studied in many works among 
which is [1] that gives a complete review of different co- 
operative strategies and new capacity theorems for the relay 
networks. In [2], a two relay network with stochastic links and 
no interference at the relays and destination is considered for 
which the capacity of some special cases have been obtained. 

Single relay network with orthogonal components was first 
studied in [3]. In [4], the capacity of discrete memoryless 
(DM) parallel two relay network with orthogonal components 
under specific conditions is derived. Moreover, the capacity of 
DM relay network with orthogonal components is obtained in 
[5]. Recently, the single relay network with orthogonal source 
components in the presence of noncausal CSI at the source 
has been considered in [6] that proposes the capacity of state- 
dependent Gaussian single relay channel with noncausal CSI 
at the source for two special cases. 

In this paper, we study the N relay network with orthogonal 
components and noncausal CSI at the source. First, we extend 
the lower bound derived in [5] for DM relay network with 
orthogonal components to the case in which the noncausal 
CSI is available at the source. Next, We propose an upper 
bound on the capacity of this model for the case in which just 
the message intended for the destination is encoded against 



the noncausal CSI by the source. In continue, it is shown that 
these lower and upper bounds coincide for two special classes 
of Gaussian relay network with orthogonal components and 
noncausal CSI at the source. 

The paper is organized as follows. In section I, the network 
model is introduced. In section II, Our main results including 
the lower and upper bounds and special capacity results for the 
Gaussian model is presented. Finally, the paper is concluded 
in section V. 

II. Preliminaries 

A discrete memoryless (DM) relay network with non- 
causal CSI at source is a channel with iV + 2 termi- 
nals consisting of a source (node 1), N relays (node 
i G T = {2,...,A^ + 1}) and a destination (node 
N + 2). The model is characterized as {Xi x x . . . x 
A'_/v+i,p(y2,2/3,---,2/7V+2|s,a;i,a;2,...,a;jv+i),3^2 x 3^3 x 

• • ■'XyN+2) with {xi,X2, ■ ■ . ,XAr+i,S,?/2, ■ • -,2/^+2) G (XiX 

X ... X Xn+1 X 5 X 3^2 X 3^3 X ■•■ X yN+2) where Xi 
and Xi are the channel input alphabets at the source and relay 
i, respectively, over which the channel input random variables 
Xi and Xi take value. Also, yN+2 and 3^; are the channel 
output alphabets at the destination and relay i, respectively, 
over which the channel output random variables y/v+2 and Yi 
take value and these output random variables are governed by 
the input random variables and the channel state S. 
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code for the relay network with noncausal CSI at 



the source consists of a set of integers >V = {1, . . . , 2"^}, one 
encoder / : >Vx5" — > A"" and N sets of functions {rp^i}"^i 



at the relays, such that Xp 



rps{yp,i,yp,2, ■ ■ . ,2/p,i-i), p e 



T, where Xp,i is the i'th component of the relay p codeword 
Xp = {xp^i, ...,Xp^n) and a decoder g : 3^JJr+2 — > W at 
the destination. The average error probability for this code 
is defined as = EsiP[g{y'}f+2) t^W]). A rate R for the 
relay network with noncausal state at the source is said to be 
achievable if there exists a sequence of (2"^, n) codes such 
that P" < e for any e > and sufficiently large n. Also, the 
capacity C of this channel is defined as the supremum of the 
set of achievable rates. 
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Fig. 1. Relay network with orthogonal components and noncausal state at 
the source. 



A DM relay network with orthogonal components and 
noncausal CSI at the source as it is shown in Figure 1, is 
a DM relay network in which the source and relay i's input 
alphabets Xi and Xi are divided into iV + 1 and N orthog- 
onal components {Xii}i(zru{N+2} and {Xij}j(,ru{N+2},j^i, 
respectively. In this case, the channel probability distribution 
function can be written as ([5]) 

P{{yj}je{N+2}uT\s, {^iJ}^e{l}urJe{N+2}UT.J^^) = 

JJp(yi|s,{a;jOje{i}ur) {xij}je{N+2}uT,m) 
ieT 

P{yN+2\s,{Xi^N+2}ie{l}UT) (1) 

with i{^ij'yj}je{N+2}ur,ie{i}uT,j7ii) ^ 

^je{N+2}uT,ie{i}ur,j^i '^^i nje{jv+2}ur -^J- 

III. Main Results 

In this section, first, a lower bound on the capacity of relay 
networks with orthogonal components and noncausal CSI at 
the source is derived and then an upper bound on the capacity 
of a special class of this network is obtained in which just 
the source component intended for the destination is encoded 
against the noncausal CSI. Next, we establish the capacity for 
two special cases of the Gaussian structure of the network 
model . 

Proposition 7 (modified version of [5, Theorem 1]); A lower 
bound on the capacity of DM relay network with orthogonal 
components and noncausal CSI at the source is given by 



C > max min_| + + R^^ + 



(2) 



where 

£{N+2}UT.J^ i) (3a) 
I^2,ri=H{^ji}jeM'''Xi^Xij}jeTU{N+2},j^i,{^ji}j(^s)0^) 

-^M ~ I{{^kN+2}keM<'',YN+2\{XkN+2}kes) (3c) 



= I{Ui ■,Yn+2 ]{XkN+2}keT) - I{Ui;S\{XkN+2}kerlM) 

with Ad" = T\ A4 and the maximum is taken over joint 
probability distributions of the form 

^'^^^ n n PiXij)p{Ui,Xu\{Xik}keTU{N+2},jjii) 

ier je{N+2}uT,j^i 

p{ui,XiN+2\s, {XiN+2}ier)p{yN+2\s, {XjN+2}j€{l}ur) 

n PiVil^^ {xij}je{N+2}uT, {xji}je{i}uT,m) j ■ (4) 
ieT ' 

Proof: This rate can be easily achieved if we combine the 
coding techniques explained in [5, Appendix 2] for a relay 
network with orthogonal components with the well-known 
Gelfand-Pinsker coding for encoding components of the source 
message intended for the relays and destination against the 
noncausal CSI at the source. The proof is omitted for brevity. 

Theorem 1: An upper bound on the capacity of DM relay 
network with orthogonal components and noncausal CSI at 
the source is given by 

C < max min | ^ (i?^,., + Rlrd + + ^Id] (5) 
where 

Rl^.^ = I{Xu;Y,\S, {X,a,er,,#. (6a) 

R2,r, = I{{^3i}]eM<']Yi\{Xji}jeM.]^i) (6b) 
-^M ~ I{{^iN+2}ieM''\YN+2\{XiN+2}ieM) (6c) 

R^,d = ^jv+2|5', {Xj;v+2}ier) (6d) 

and maximization is over all joint probability of the form 

p{s)p{ {a;ij}jeru{JV+2}, {xij}jeT ]p{x-in+2\s, {xiN+2}ieT) 

n Piyi\^' {Xji}je{l}UT,jjii)p{yN+2\s, {XjN+2}je{l}UT)-0) 

leT 

Proof: Refer to Appendix. 

Now, we consider a special case of Gaussian relay networks 
with orthogonal components and noncausal CSI at the source 
in which the source sends the message intended for the des- 
tination by using dirty paper coding (DPC) and the messages 
sent to the relays are independent of the CSI. In this model, 
the channel outputs for the relay j and destination at time 
instant i is described as 

Yj^i= Xr,,i + Zj, „ j e r (8a) 

re{l}ur,r7tj 

^iV+2,j= -'^rJV+2,i -V ^JV+2,i (8b) 

re{i}ur 

where Si is the additive Gaussian noncausal CSI random 
variable available at the source with zero mean and variance 
Q. Also, Zj/s, j E T U {N + 2}, are mutually independent 
additive Gaussian noise random variables with zero mean and 
variance Nj which are independent of the S and the channel 
inputs. Power constraints on the components of the source and 
relays codewords are as 



^ n^l^A < nPrj , r e { 1} U r, e r U {TV + 2} , r ^ J. (9) 

Now we use Proposition 1 and Theorem 1 to exploit 
the capacity of the Gaussian relay network with orthogonal 
components in (8). 

Theorem 2: The capacity of Gaussian relay network with 
orthogonal components of (8) is given by (10) at the bottom 
of the page where the maximization is over all p\s € [—1,0] 
and pij e [0, 1] with 

Proof: We use a proof which is similar to the 
one in [6] in some steps. Converse part: We first 
fix a distribution as in (7) for the random variables 

{S, {Xij}i(z{i}yjr,je{N+2}\jT.j^ii {^"}ie{Af+2}ur) where 

E[X2.] = p,j- < p,^-; i e {1} u r, j e {TV + 2} u r,,? ^ I, 



lN+2-tiN+2, 



i G T, and 

E[Xijv+2<S'] = pisJ PiN+2Q- Now, we calculate the terms 
in (5), 

jyu I T>u 
1 r ■ ~^ r ■ 

= h{Yi\{Xji}j^M,3¥^i) - KYi\{Xji}jer) 

+h{Yi\S, {Xji}jeT,jTti) - h{Yi\S, {Xji}j^{iyur) 
= h{ ^ Xji + Zi\{Xji}j^M) - h{Xu + Zi\{Xji}j^r) 



+h{Xu + Z,\S, {Xjjjer) - h{Zi) 
ZA - h{Zi) 



(a) , 

je{i}uM 



Xji 



(fc)l 



1 



(12) 



where (a) follows since removing condition does not re- 
duce entropy and (6) follows since the maximum entropy is 
achieved if ({-^ji}je{i}ur,j>ti, -^0 are jointly Gaussian. Next, 
for the other two terms in (5) we have 

iN+2}ieM''lYN+2\{XiN+2}iGM) 
— I{{XiN+2}ieM'=', S\{XiN+2}ieM) 
+I{{XlN+2}: yN+2\S,{Xi]y-^.2}^£T) 
= I{{XiN+2}ieM''',YN+2\S, {XiN+2}ieM) 
—I{{XiN+2}ieM''', S\Yn+2, {XiN+2}ieM) 
+I{XiN+2', YN-^-2\S,{XiN^2}ieT) 

= h{YN+2\S, {XiN+2}ieM) — h{YN+2\S, {Xi7v+2}ie{i}ur) 
+h{S\YN+2, {XiN+2}ier) — h{S\YN+2, {XiN+2}ieM) 



= h{YN+2\{XiN+2}ieM) + HS\Yn+2, {XiN+2}ieT)) 

-h{Zi) - h{S\{XiN+2}^eM) (13) 

where (c) is deduced since {-^iAr+2}ieM'= and S are indepen- 
dent. Moreover, we have 

h (YM+2\{XiN+2}izM) 

<-log(27re)f 

PiN+2 + '^Pli\J P\N+2PiN+2 

^ie{l}UA^= ieM= 

+ '^Pis\l P1N+2Q -E pIAn+2 + Q + Nn+2\ (14) 

where {d) holds if YAr+2 is jointly Gaussian with 

{X,,N+2]ieM- The second term h{S\YN+2AXiN+2}ieT) 
in (11) is maximized if {S,YN+2,{XiN+2}ieT) are 
jointly Gaussian. Suppose that S{YM+2T{XiN+2}ieT) = 
Y.jeT^]^]N+2+li{XiN+2+S+Zi^+2) is the linear MMSE 
estimation of S given (i^jv+2) {-'^ijv+2}i£r)- Thus, we can 
write 

h{S\YM+2, {XiN+2}ieT) 

=h{S — S{Yn+2, {XiN+2}ieT)\YN+2, {XiN+2}ieT) 
<h{S - 7jXjN+2 - 7l(^lJV+2 + S + Zn+2)) 



=-log(27re)E 



('S'-y^7j^jiV+2 -7l(-'^lAf+2 + <S' +-^JV+2))^ 



=-log(27re)( 0-7i(pi,^Piw+2Q + Q) 



(15) 



and 7i, i G {1} U T, is computed using the orthogonaUty 
principle as 

(pis^ P1N+2Q + Q) 



71 = 



7r = 



(1 - Ej(^rPij)PiN+2 + Q + Nn+2 + 2pu\IPlN+2Q 



-pir\l^^{pis\JPlN+2Q + Q) 



(1 - E,er + "3 + Nn+2 + 2p 

Is \J P1N+2Q 

where r gT. After substituting (16) in (15) we have ^^^^ 

h{S\YN+2, {XiN+2}ieT) 

Pijv+2(1 - pL - E,er Pij) + Nn+2 



Q 



<- 



=(17) 



{l-EjeTph)PiN+2+Q + NN+2 + 2pu\/Pl N+2Q 

Therefore, -R"d + -^2,ci obtained by replacing (14) and (17) 
in (13) and hence 



n ■ f ^ ^ ^ H , EjeiuM<^^i»x 1, A , PiN+2{'i-- Pis-J2jeTPij) 
C = ma.mm|^-log(l + )+-log[l + 

2 n (1 - pL - E,er P?i)^iiv+2 + (x/Q + Pi.v/A]^)2 + TVjv+2. ^ 



1 / EjeM' 
+ -logll + 



■i\J PjN+2 + Plj\J P1N+2Y 



(18) 



wher e A = (1 - p?^ _ Y.jeT plj)P^N+2 + (\/^ + 

Pis'\J Pin +2)"^ + -^^+2- It can be readily seen that the terms 
obtained in (12) and (18) are increasing functions of Pji for 
z G T, j G {1} UT, and PjN+2 for j € T, and as a result the 
upper bound (5) is maximized for the maximum value of these 
parameters i.e. Pji = Pji and PjN+2 = PjN+2- Also, it has 
been shown in [6] that the second term in (16) is an increasing 
function of PiJV+2 so, the maximum value of this parameter 
i.e. PiN+2 maximizes this term. Finally, as it can be seen if 
we let Pig € [—1,0] and pij S [0, 1] the two terms in (18) 
would take their maximum value. As a result, we showed that 
if {S,{X,j,Zj},^[i}ur.3eru{N+2}.j^i) are jointiy Gaussian 
the the upper bound (5) is maximized. 

Achievability part: We make use of Proposition 1 along 
with the following assumptions. We let X^j's for i G {1} U 
T,j S {N + 2} U T, i 7^ i to be jointly Gaussian random 
variables A/'(0, Pij). Also, we assume that Xi^'s, j e T, and 
's, i E T;j E {N + 2} U T , i / j, are all independent of 
each other and independent of the state S. The input random 
variable X1N+2 is jointly Gaussian with S and is independent 
of X ij's for i € {1} U r, j eT, i ^ j with E[Xin+2S] ^ 

Pls\/QPlN+2 and K[XiN+2XjN+2] = Pij PlN+2PjN+2, 

j G T, where pis € [—1,1] and e [—1,1]. Next, we set 
Uj = Xij, j e T, and 

^lN+2 = 2_^Plj\j -5 -^jN+2-tPU\l 7; + -^lN+2 



jer 



jN+2 



Q 



Ui=X 



lN+2- 



Q 



(1-pL - EjeTPij)PiN+2 + 



s 



(19) 

where Xijv+2 ^ {O, {1- plg-J2jeT Pij)PiN+2) and it can 
be easily verified that it is independent of S and {Xjjv+2}jer- 
So YAr+2 can be written as , 

Yn+2=Xin+2 + ^'(l + Pij jN+2 



-(l + Pl 




JV+2 



(20) 



With these choices of random variables, we have 

Pl,ri + -^2,ri 

^—I{Xu\ Yi\{Xij}j^{iq+2}UT,ji^i, {Xji}jeT,jTn) 

-^lii^ji] jeM<=\Yi\{Xij} j(zru{N+2},ji^i-, {Xji}j<^M,j^-i)) 

=l{{Xji],j^{mjMc;Yi'^Xij}j^ru{N+2},jjti,{Xji}jeM,jTti)) 

(/) 1 l_„ A ^ EjelUA^- Pji 



2^°S'V 



N,, 



(21) 



where (e) holds because Xu is independent of S in the special 
case (8) and (/) follows from (12) with Pji being replaced 
by Pji and going through the same steps as in (12). 



Also, for R2 d and R{ ^ we have 

, 1 / E[X? 



,V+2J 



Xn+2 

^ V -'V7V+2 / 

P-i,d=H^i^+2\{XjN+2}jeM) - h{Y'N+2\{X jN+2} jer) 



-h(xiN+2 + E (1 + J§^)^.iV+2 



P. 



jN+2 



+ {1 + PiJ^^)S + Zn+2 



-h{XiN+2 + (1 + Pls\ ^^r^)S + Zn+2) 



Q 



-Y^gl^ I EjgA^c(V -P;W+2+PljVflJV+2)^ 



e 



(23) 



' E,erPh)PiN+2 + iVQ + 

□ 



wher e 6 = (1 - pi, - z^^gr/^y 
Pis\/Pin+2)'^+Nn+2- This completes our proof. 



The other special case of interest for which we have derived 
the capacity, is the Gaussian relay network with orthogonal 
components and no interference at the relays and the destina- 
tion and with the CSl known noncausally at the source. The 
channel outputs for this case can be considered as 



(24a) 



Yl^X,j,i + Z^, j,rer, j^r 
Yy=Xij,^ + S^ + Zl^, je{N + 2}UT (24b) 



Y, 



[r] 



N+2.i 



—XrN+2,i + Z 



(r) 

N+2,i 



, r eT 



(24c) 



where YJ'.^ and ijv+2 % the channel outputs at the relay 

j and the destination due to the the channel inputs Xrj and 
XrN+2.i from relay r 7^ j, respectively. Also, Z^^^'s, r G 
T,r ^ j, are the additive Gaussian noise random variables at 
node j, j e {N-\-2}yjT, with zero means and equal variances 
Nj all of which are mutually independent and independent 
of CSI and the channel inputs Xx = {Xii}ieT and Xr = 

{Xrj}j^rL>{N+2}- 

Proposition 2;(modified version of [5, Theorem 1]) An 
upper bound on the capacity of DM relay network with 
orthogonal components and noncausal CSI at the source is 
given by 

C <maxinin <^ ^ /({Xji}jg{i}uA^c; Fil^, {Xjijje^) 

+I({XiN+2}ie{l}uM'''XN + 2\S,{XiN+2}ieM)^('^5) 

for some joint probability of the form 

p{s,{xi3}te{i}urjeT.j^t)piyN+2\s,{xjN+2}je{i}uT) 

'[lp{y2\s,{xji}j^{i}ur,j^i) (26) 
ier 

Proof: This bound has been proved in [5] for the 
case of no CSI (S = 0) using the cut-set upper bound 



iaillMeT{H{Xi}i(z[i}uM'AYi}ie{N+2}UMAS, {Xtji^M")} 

and by setting Xi = ({^ij}jg{jv+2}ur,j7ti). « G {1} U T. 
The proof for our model follows the same steps as in [5] with 
this modification that for our model in (24), we have S ^ 
and {Xij}j(,{N+2}uT {Xji}jeT,jj^i Yj- So, the proof 
is straightforward and is removed for brevity. 

Theorem 3: The capacity of the Gaussian relay network 
with orthogonal components, no interference at the relays and 
destination with CSI known noncausally at the source is 



1 /3*P- 
C = max min i ( - loe(l H — tt^) 

Mcr\ ^ \ ^ 2 TV, ^ 



log(l + ^^^ 



(27) 



Ar+2 



where the maximization is over all e [0, 1] subjected to 

Sie{jv+2}ur,j5^fe 1^1 — ^ ^ {1} ^ 
Proof: The achievabiUty part can be proved by setting Uj = 

0^ Pi 

Xij + pjp _|_jv Proposition 1 and applying the model (24) 

with xlj ~ .^(0, PiPk) for j e{N + 2}UT,kG{l}UT 
which are aU independent of S. Also, Xu's, i S T, are all 
independent of Xij's for i eT, j & {N + 2}uT,j ^ i. So, 
we have 



Rir, = \ l0g(l + 4... = E ^ 



JV+2i 



R 



1 oiv +:^; p 



(28) 



/se{i}uAi= 



For the converse part we make use of the result in 
Proposition 2, following the same steps as in the con- 
verse part of Theorem 2. First, we fix a distribution on 

{S, {-'^ij}ieiur,je{iv+2}ur,i7ij) {X7}je{Jv+2}ur) as 

P{S) n PiXi3)'[\_P{yi\s,{x3i}3^{l}UT,3^i) 

ie{i}ur,je{iv+2}ur »6r 

P{yN+2\s,{XjN+2}j^T,yj{l}} (29) 

that satisfies E[X2.] = < -Py, « G {1} U T, j G {iV + 

2}ur. 

Next, we calculate the first term in the relation (25). We 

can write 

I {{X'ji}je{i}yjM<']Yi\S-, {Xji}jeM,]^i) 

= h{{Xri + Z\ ''}rG{l}UM',r^i,{Zi }reM,r=^i\Si{Xji}jeM) 

-h{{zr>}r 



(a) 



<h{{Xr^ + Zl''>}re 



h{{zt^re{i}ur) < 



ib) 



E + (30) 



where (a) holds since removing condition does not reduces 
entropy and (6) holds because of the fact that the entropy 

{UuM" y{Zi''''}reM,r^i) IS maximized if 

{{Xri}re{i}uT, {-^1 }re{i}ur) are jointly Gaussian. 



Now, the second term in (25) is calculated. We have 

I {{XiN+2}ie{l}UM'=i Yn+2\S, {XiN+2}ieM) 

=h{{XrN+2 + Z^^_^_2}re{1.}UM'=-:{ZN+2}reM\S,{XiN^2}ieM) 

-/i({Z^|2}re{i}ur) 



(c) 



<h{{XrN-t-2 + -^Ar+2}''e{l}UX<^7 {■^]v+2}'"6A4) 

-log(l + ^ 



, . (d) 1 

-h{{zP+2}re{i}ur) < E 2^°S(1 

re{l}UX = 



JV+2 



) (31) 



where (c) is deduced in the same way as (a) 
and (d) holds since the entropy h{{XrN+2 + 
{i}\jM'=Az'"M+2}reM) is maximized if 

({-'^ri}re{i}uA^-,{-^i'^^}re{i}ur) joiutly Gaussian. 

Thus, we have 



C < min , . 

MCT[ ^ ^ 2 

^ i&M re{l}uM 



E 



re{l}UM' 



-'-1 /I , PrN+2 ■ 
^l0g(l + -TT , 

2 -/Vjv+2 



(32) 



Right hand side of the above relation is an increasing 
function of Pij then the maximum happens for the values 
Pij = Pij. This completes the proof of Theorem 3. □ 

Corollary: Our results in Theorem 2 and 3 reduces to that 
of obtained in [6, Theorem 7] and [6, Corollary 2] as special 
cases when we assume just one relay i.e. T = {2}. 

IV. Conclusion 

In this paper, the relay network with orthogonal components 
and noncausal CSI at the source was investigated. For the DM 
model, a lower bound and an upper bound was developed 
which met for two special cases of the Gaussian structure of 
the model. The fist case was the one in which just the intended 
message for the destination was sent using the CSI and the 
other case was the one in which all of the source components 
were encoded against the CSI but there existed no interference 
at the relays and destination. 
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Appendix 

Proof of Theorem 1: 

We prove that for any (2"^, n) code with average probability of error P" — > as n — > oo, the rate R must satisfy (5). 
Fano's inequaUty, we have H{W\{YP}i^s-=u{N+2}) < H{W\Y^^2) < 1 + nEe„ = 

nR = HiW) = I{W; {y/}ie5cu{iv+2}) + H{W\{Y;},^scu{n+2}) 

<I{W;{YJ^},^s^u^N+2}) + n6n 

= liW, 5"; {r/},e5=u{A^+2}) - HS"; {Yp}j^s^u{N+2}\W) + n5n 

n 

= I{W, 5"; {Yj,i}jes^u{N+2}\{Y;~'}jes^u{N+2}) - H{S") + H{S"\W, {Yp}jes^u{N+2}) + nd^ 

i=l 

= X] [^i{Yj,i}j&S-=U{N+2}\{Yj^^}j(zs-=u{N+2}) " H{{Yj^i}j(,s-=U{N+2}\W,S",{Yj^^}j(zs.u{N+2}) 
j=l ^ 

- H{Si) + H{Si\S'-\W, {F/},e5cu{JV+2})) + n<5„ 
^ 2^ ( H{{Yj^i}j^s'=u{N+2}\{Xjk,i}jeS'=,ke{N+2}uT) - H{{Yj^i}j^s'=u{N+2}\Si, {-'^j;c,i}je{i}uT,fce{Jv+2}ur) 



-ii—l 



-H{Si)+H{Si\S'-\W,{Y]'},^S'^[N+2}) 

n ^ 

id) 



( 

{i}u5,fee{iv+2}ur) Si\ {5G',ibe5<^u{Jv+2}|{^j/s,i}je5°,/se{Jv+2}ur) 

- HiSi) + W, {l^"}jeru{JV+2}> {-^jfc,i}je{i}u5",feeru{Ar+2})^ 

(e) " / 

{i}u5,/se{jv+2}ur) 'S'i; {^■,j}je5'^u{iv+2}|{-^jfe,i}je5°,fee{iv+2}ur) 

i=i ^ 

- -H'('S'i) + -ff ('S'j|{lj,j}jeru{iv+2}, {-'^jfc,i}je5°,/seru{Jv+2})^ 

{l}U5,fee{Ar+2}ur) -S**; {^i,i}i6S<=U{Af+2}|{-'^ift,i}jeS=,A;e{iV+2}ur) 

j=l ^ 

- l{Si\ {^•,i}je5=u{Jv+2}! {-'^jfc,i}jer,fceru{Jv+2})^ 

je5,fee{iv+2}ur; {^■,i}je5=u{iv+2}|{-'^jfe,i}je5°,/se{jv+2}ur) 

j=i ^ 

+ -f({-'^lfe,i}fee{Ar+2}ur; {y7,0ie5<=U{Ar+2}|'S'i, {Xj7;,j}jg7-_feg{Ar_,_2}ur) - i{Si\ {X iaT ,k(ir\J{N Jr2})^ 

je5,fee{iv+2}ur; {^■,j}je5<=u{iv+2}|{-'^jfe,i}je5°,fce{jv+2}ur) 

i=l ^ 

+ -'^({-^ifc,i}fce{jv+2}ur; {^■,i}je5<=u{Jv+2}|'S'i, {-'^jfc,i}jer,fee{iv+2}ur)^ 

0) 

^ -^({-'^jfeb6S,fee{Af+2}ur; {X7}iGS<=U{Ar+2} |{-'^ife}je5",fee{Af+2}ur) 

+/({Xifc}fcg{jv+2}ur; {^j}je5<=u{jv+2}|5', {-'^jfc}jer,fee{iv+2}ur) 

= J({Xjfe}jg5,feg{jv+2}ur; {^j}je5<^|{-'^ife}ie5=,fcG{Ar+2}ur) 

+^({-'^jfe}iG5,fce{Ar+2}ur; ^Ar+2|{yj}jes^, {-^jfe}jeS=,feG{Af+2}ur) 

+-?^({-'«^lfe}fcG{W+2}ur; {^j}iG5<=|<S', {-'^jfc}jer,fcG{Af+2}ur) 

+-'"({-'^ife}fee{iv+2}ur; i^iv+2|<S', {y,}je5<=, {-''^ifc}ier,fce{w+2}ur) 

(m) 

< ^({^j}je5=l{^jfe}je5=,fce{jv+2}ur) - ^({^j}je5=|'S', {Xjfc}je{i}ur,fce{jv+2}ur) 



+H{YN+2\{Yj}j^s-=,{Xjk}jeS'=,ke{N+2}uT) - H{Yn+2\S, {Yj}jes-= , {Xjk}je{i}uT,ke{N+2}uT) 
^= H{{Yj}j^s<=\{Xjk}jeS'',ke{N+2}uT) - 5Z -^(^^1'^' {Xkj}ke{i}us, {Xkj}keS'=,{Xjk}ke{N+2}uT,kjij) 

+H{Yi^^2\{Yj}j(zs<=, {Xjk} jes" ,ke{N+2}ui — H{Yn^2\S, {-'^fcAf+2}feG{i}ur) 

< 2^ H{Yj\{Xkj}keS'=,kjij,{Xjk}je{N+2}uT,k^j) - H{Yj\S, {Xkj}ke{i}us, {Xkj}keS'=,{Xjk}ke{N+2}uT,kjtj) 
jes- 

+H{YN+2\{XjN+2}jes<:) - H{Yn+2\S, {-^fcjv+2}fce{i}ur) 
(9) ^ 

< 2^ H{Yj\{Xkj}keS'=,kjij,{Xjk}je{N+2}uT,k^j) - H{Yj\{Xkj}k&T,kyij , {Xjk}je{N+2}uT,k^j) 

jes- 

+ X! ^j\{^kj}keT,kjtj, {Xjk}je{N+2}uT,kjij) - H{Yj\S, {Xkj}ke{i}us, {Xkj}keS'=,{Xjk}ke{N+2}ur) 

+H{YM+2\{XjN+2}jGS'') — H{YN+2\{XjN+2}jGT) + H{S, YN+2\{XjN+2}ieS'') — H{Yn+2\S, {-^fcJV+2}feG{l}ur) 
— X! ^^{^kj}k(iS,k^i;Yi\{Xkj}k£S'^,k^j,{Xjk}j(i{N+2}\jT,k^3) ~^ H{S\{Xki}k(iT,k^ii{Xjk}j(i{N+2}\jT,k^3) 

+ ^ H{Yj\S, {Xkj}keT,kT^j, {Xjk}je{N+2}uT,kjij) - H{Yj\S, {Xkj}ke{i}us, {Xkj}keS'=,{Xjk}ke{N+2}ur) 

+^i{XjN+2}jeSi YN+2\{XjN+2}jeS^) + I{XiN+2', Yn+2\S, {XkN+2}keT) 

= XI ^({■^kj}kes,kjij;Yj\{Xkj}keS'=,kjtj, {Xjk}je{N+2}uT,kjij) + IiXij;Yj\S, {Xkj}keT,kjtj, {Xjk}je{N+2}uT,kjij) 

+I{{XjN+2}jeS]YN+2\{XjN+2}jeS':) + I{XiN+2] 5^^+215", {XkN+2}keT) 

where (a) follows by Fano's inequality; (6) follows since 5" is an i.i.d process and that it is independent of message 
W; (c) follows since Xj = {-^jfc,j}je5Sfce{JV+2}ur is a deterministic function of Yj~^ and that {W, S"' ,Y-'~^) — )• 
{Si,{Xjk,i}jes-,ke{N+2}) Yjy, (d) follows since Yj~'^ (S"~\ VK, {Yj"}jeru{Af+2}) 5^; (e) follows since removing 
conditions does not reduce entropy; (/) and {i) follows since Si is independent of {Xjk}jeT,keTu{N+2}', (j) follows by the 
common arguments for removing n letter characterization presented in most of the upper bounds which is also explained in 
[6, Theorem 4]; (m) follows since removing condition does not reduce entropy; (n) follows from the distribution in (7); (q) 
follows since removing condition does not reduce entropy and the fact that the joint entropy of random variables is not greater 
than the entropy of each random variable; (r) follows in the same way as (i). 



